The present invention relates to a picture processing apparatus and method and, more particularly, to a picture processing apparatus and method which receive picture data and change information as information for indicating a temporal change in the picture data, thereby to generate picture data for display.
The technology of digitizing a picture to obtain digital picture data is an important technology for display or transmission of picture data in broadcasting, picturephones, videoconferences, computer network systems and the like. As for storage of the digital picture data, mass storage media such as DVDs are becoming widespread and coded moving picture data compressed according to MPEG1, MPEG2 or the like is usually stored in these media.
The data stored in such a medium is reproduced by a reproduction apparatus including a DVD player when the medium is a DVD. In the case of reproduction using data transmitted in a network system or the like, the normal reproduction of displaying pictures similar to original moving pictures is exclusively performed. On the other hand, in the case of the reproduction by the above reproduction apparatus, various trick reproduction functions are mostly provided for users"" convenience. As examples of the trick reproduction functions, there are reverse reproduction of reproducing a picture in the direction opposite in the time series to that of an original moving picture, high-speed reproduction of reproducing a picture so as to have a higher-speed motion than that of an original moving picture, and high-speed reverse reproduction of reproducing a picture at higher speeds and in the opposite direction.
For example, in a case where compressively coded data stored in the DVD is reproduced by the reproduction apparatus including the DVD player, when the normal reproduction is performed, the apparatus expansively decodes the compressively coded data successively and outputs (displays or the like) the data. On the other hand, when the high-speed reproduction is performed, the apparatus usually outputs (displays or the like) only part of data, thereby to realize a high-speed display. The compressively coded moving picture data is generally processed with one screen, i.e. one frame, as a unit. Therefore, while respective frames are successively processed to display pictures in the normal reproduction, a frame-drop processing of thinning out part of frames is performed in the high-speed reproduction. For example, when picture data consists of 25 frames per second, and only one frame among the 25 frames is processed and output, 25-time speed reproduction is performed.
FIG. 15 is a diagram for explaining the normal reproduction and the high-speed reproduction for moving picture data taking a frame as a unit. Each of F1 to F13 shown in the figure illustrates a picture of each frame included in output pictures of the normal reproduction. These pictures are to be reproduced in this order according to the time series. When the reproduction apparatus normally reproduces this data, data of F1 to F13 is successively output in a cycle according to the television signal standards such as NTSC or PAL.
The arrows in the figure indicate the order of outputting frames in the high-speed reproduction of quadruple-speed. In this case, after an F1 frame is output, an F5 frame is output and then F9 and F13 frames are successively output. Therefore, only one frame is output every four frames, whereby the high-speed reproduction of quadruple-speed is realized.
In the case of reverse reproduction, frames are successively reproduced from F13 to F1 in the reverse order to the time series. In the case of high-speed reverse reproduction, also one frame is output every four frames, i.e. F13, F9, F5, and F1 are output in the reverse order to the time series, whereby the high-speed reverse reproduction of quadruple-speed is realized.
As described above, these trick reproduction functions are popular in the reproduction apparatus. However, also in the case of transmission of picture data, when the transmission at a low bit rate is required, frames are thinned out in a similar way to the high-speed reproduction, thereby transmitting output data.
As described above, prior art picture processing apparatuses thin out frames to output (display or transmit) pictures, thereby realizing the high-speed reproduction. However, since frames which are targets to be thinned out are not output at all in this processing, information about motions of moving pictures, which is included in the frames to be thinned out is completely lost.
Assume that an object has a motion from the right to the left in the frames F2 to F7 of the picture data shown in FIG. 15. When the high-speed reproduction of quadruple-speed is performed for this picture data, the motion of the object from the right to the left is not transmitted at all by information transmitted with only the frame F5. Therefore, users who utilize a displayed result of the high-speed reproduction cannot obtain information as to how the object moves or whether the object is not moving but stops.
In addition, when the prior art picture processing apparatus performs the high-speed reproduction by thinning out frames, output frames are discontinuous. Therefore, only displays which make the users feel disorder are obtained.
Further, when the prior art picture processing apparatus performs the high-speed reproduction by thinning out the frames, the users cannot easily obtain information as to whether the normal reproduction or high-speed reproduction is performed, from only the displayed result. That is, not only when the degree of thinning is low (relatively low speed) but also the degree of thinning is high (relatively high speed), although the display makes the users feel disorder as describe above, the users cannot easily know whether the high-speed reproduction is performed or moving pictures for producing special effects are normally reproduced.
Also when the data is transmitted by thinning out frames as described above in the transmission at a low bit rate, the same phenomenon occurs when the data is displayed and utilized at a transmission destination.
It is an object of the present invention to provide a picture processing apparatus and method which can transmit information about motions included in original pictures also in the case of the high-speed reproduction.
In addition, it is an object of the present invention to provide a picture processing apparatus and method which can realize good displays having less disorder due to discontinuous displays also in the case of the high-speed reproduction.
It is an object of the present invention to provide a picture processing apparatus and method which allow the users to easily know whether the high-speed reproduction is performed or not, from a display screen or the like.
It is an object of the present invention to provide a picture processing apparatus and method which transmit information of motions, produce no disorder, and allow the user to easily know reproduction states, also when it transmits data by thinning out frames in a similar way to the high-speed reproduction.
Other objects and advantages of the present invention will become apparent from the detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the spirit and scope of the invention will be apparent to those of skill in the art from the detailed description.
According to a 1st aspect of the present invention, a picture processing apparatus which receives picture data and change information as information for indicating a temporal change in the picture data and generates displayed picture data, comprises a motion generation unit for performing weighted addition for pixels constituting the picture data on the basis of the change information, and generating the displayed picture data. Accordingly, the displayed picture data includes the motion information and a display result having a sense of motion can be obtained. Therefore, a display having less disorder relative to the prior art picture processing apparatus can be realized, and users can easily know a reproduction state because the motion information is added to the picture data.
According to a 2nd aspect of the present invention, in the picture processing apparatus of the 1st aspect, the motion generation unit uses the change information which is previously added to each unit picture data, thereby to perform the weighted addition for pixels constituting the unit picture data. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 3rd aspect of the present invention, in the picture processing apparatus of the 1st aspect, the motion generation unit generates, on the basis of plural pieces of the change information which is previously added to each unit picture data, change information corresponding to processing unit picture data which is a smaller unit than the unit picture data, and uses the generated change information corresponding to the processing unit picture data, thereby to perform the weighted addition for pixels constituting the processing unit picture data. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 4th aspect of the present invention, in the picture processing apparatus of the 2nd aspect, the motion generation unit comprises a move amount and weighting factor decision unit for deciding a move amount and weighting factors for each pixel, from the change information; a processing target pixel decision unit for deciding a pixel as a processing target on the basis of the move amount; and a weighted addition unit for performing the weighted addition for pixel data of the pixel as the processing target, by using the weighting factors. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 5th aspect of the present invention, in the picture processing apparatus of the 3rd aspect, the motion generation unit comprises a motion vector interpolation unit for generating, on the basis of plural pieces of the change information which is added to each unit picture data, change information corresponding to the processing unit picture data which is a smaller unit than the unit picture data; a move amount and weighting factor decision unit for deciding a move amount and weighting factors for each pixel, from the change information corresponding to the processing unit picture data; a processing target pixel decision unit for deciding a pixel as a processing target, on the basis of the move amount; and a weighted addition unit for performing the weighted addition for pixel data of the pixel as the processing target, by using the weighting factors. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 6th aspect of the present invention, in the picture processing apparatus of any of the 1st to 3rd aspects, the motion generation unit obtains change information transition information for indicating a temporal change in the change information, and performs the weighted addition for the pixels constituting the picture data on the basis of the obtained change information transition information, thereby to generate the displayed picture data. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 7th aspect of the present invention, in the picture processing apparatus of the 4th or 5th aspect, the move amount and weighting factor decision unit comprises a change information storage unit for storing the input change information, and decides the move amount and the weighting factors for each pixel on the basis of plural pieces of change information at different times, which change information is stored in the change information storage unit. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to an 8th aspect of the present invention, the picture processing apparatus of any of the 1st to 3rd aspects further comprises a reproduction information input unit for inputting reproduction information which indicates a display state of the displayed picture data, and the motion generation unit generates the displayed picture data by using the reproduction information. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 9th aspect of the present invention, the picture processing apparatus of any of the 1st to 3rd aspects further comprises a threshold judgement processing unit for comparing the input change information with a threshold, and generating second change information on the basis of a result of the comparison, and the motion generation unit generates the displayed picture data by using the second change information. Accordingly, when the camera shake having a micro motion vector, or a picture or object having almost no motion vector is included, the motion information is not added. Therefore, the risk that the picture including the camera shake is erroneously recognized as the motion of the picture can be avoided. Since unnecessary motion information is not given to the picture, a picture processing apparatus can transmit the motion of an original picture and obtain a good picture including less disorder or displeasure.
According to a 10th aspect of the present invention, the picture processing apparatus of any of the 1st to 3rd aspects further comprises a motion vector processing unit for multiplying a value of the input change information by a predetermined factor and generating second change information, and the motion generation unit generates the displayed picture data by using the second change information. Therefore, a picture having an emphasized motion information added thereto, or contrary a picture having a suppressed motion information added thereto can be generated. Accordingly, a picture processing apparatus can generate and display a picture to which more dynamic presence is added, or motion information is added with the presence being reduced.
According to an 11th aspect of the present invention, the picture processing apparatus of any of the 1st to 3rd aspects further comprises a region judgement unit for judging a region of the picture data, as a processing target. Therefore, a region to which the motion information is added can be arbitrarily decided. Accordingly, a part in which dynamic presence is to be given can be distinguished from other parts according to provided picture data.
According to a 12th aspect of the present invention, a picture processing method which receives picture data and change information as information for indicating a temporal change in the picture data and generates displayed picture data, comprises a step of performing weighted addition for pixels constituting the picture data on the basis of the change information, thereby generating the displayed picture data. Accordingly, the displayed picture data includes the motion information and a display result having a sense of motion can be obtained. Therefore, a display having less disorder relative to the prior art picture processing apparatus can be realized, and users can easily know a reproduction state because the motion information is added to the picture data.
According to a 13th aspect of the present invention, the picture processing method of the 12th aspect comprises a step of using the change information which is previously added to each unit picture data, thereby performing the weighted addition for pixels constituting the unit picture data. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 14th aspect of the present invention, the picture processing method of the 12th aspect comprises steps of on the basis of plural pieces of the change information which is previously added to each unit picture data, generating change information corresponding to processing unit picture data which is a smaller unit than the unit picture data; and using the generated change information corresponding to the processing unit picture data, thereby performing the weighted addition for pixels constituting the processing unit picture data. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 15th aspect of the present invention, the picture processing method of any of the 12th to 14th aspects comprises steps of obtaining change information transition information for indicating a temporal change in the change information; and performing the weighted addition for the pixels constituting the picture data on the basis of the obtained change information transition information, thereby generating the displayed picture data. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 16th aspect of the present invention, the picture processing method of any of the 12th to 14th aspects comprises a step of receiving reproduction information for indicating a display state of the displayed picture data, and generating the displayed picture data by using the reproduction information. Therefore, a display result which corresponds to a state of a picture more appropriately can be obtained.
According to a 17th aspect of the present invention, the picture processing method of any of the 12th to 14th aspects comprises a step of comparing the input change information with a threshold, and generating the displayed picture data by using second change information generated on the basis of a result of the comparison. Accordingly, when the camera shake having a micro motion vector, or a picture or object having almost no motion vector is included, the motion information is not added. Therefore, the risk that the picture including the camera shake is erroneously recognized as the motion of the picture can be avoided. Since unnecessary motion information is not given to the picture, a picture processing apparatus can transmit the motion of an original picture and obtain a good picture including less disorder or displeasure.
According to an 18th aspect of the present invention, the picture processing method of any of the 12th to 14th aspects comprises a step of generating the displayed picture data by using second change information generated by multiplying a value of the input change information by a predetermined factor. Therefore, a picture having an emphasized motion information added thereto, or contrary a picture having a suppressed motion information added thereto can be generated. Accordingly, a picture processing apparatus can generate and display a picture to which more dynamic presence is added, or motion information is added with the presence being reduced.
According to a 19th aspect of the present invention, the picture processing method of any of the 12th to 14th aspects comprises steps of judging a region of the picture data as a processing target, and generating the displayed picture data by using a result of the judgement. Therefore, a region to which the motion information is added can be arbitrarily decided. Accordingly, a part in which dynamic presence is to be given can be distinguished from other parts according to provided picture data.
According to a 20th aspect of the present invention, there is provided a computer readable recording medium on which a program for causing a computer to execute weighted addition for pixels constituting picture data on the basis of change information as information for indicating a temporal change in the picture data is recorded.
According to a picture processing apparatus or method of the present invention, also when a picture is transmitted at a low bit rate, an original motion of the picture is transmitted, thereby obtaining the same effects as in the case of the reproduction time.